533 research outputs found
Convex Optimization-based Policy Adaptation to Compensate for Distributional Shifts
Many real-world systems often involve physical components or operating
environments with highly nonlinear and uncertain dynamics. A number of
different control algorithms can be used to design optimal controllers for such
systems, assuming a reasonably high-fidelity model of the actual system.
However, the assumptions made on the stochastic dynamics of the model when
designing the optimal controller may no longer be valid when the system is
deployed in the real-world. The problem addressed by this paper is the
following: Suppose we obtain an optimal trajectory by solving a control problem
in the training environment, how do we ensure that the real-world system
trajectory tracks this optimal trajectory with minimal amount of error in a
deployment environment. In other words, we want to learn how we can adapt an
optimal trained policy to distribution shifts in the environment. Distribution
shifts are problematic in safety-critical systems, where a trained policy may
lead to unsafe outcomes during deployment. We show that this problem can be
cast as a nonlinear optimization problem that could be solved using heuristic
method such as particle swarm optimization (PSO). However, if we instead
consider a convex relaxation of this problem, we can learn policies that track
the optimal trajectory with much better error performance, and faster
computation times. We demonstrate the efficacy of our approach on tracking an
optimal path using a Dubin's car model, and collision avoidance using both a
linear and nonlinear model for adaptive cruise control
Data-Driven Reachability Analysis of Stochastic Dynamical Systems with Conformal Inference
We consider data-driven reachability analysis of discrete-time stochastic
dynamical systems using conformal inference. We assume that we are not provided
with a symbolic representation of the stochastic system, but instead have
access to a dataset of -step trajectories. The reachability problem is to
construct a probabilistic flowpipe such that the probability that a -step
trajectory can violate the bounds of the flowpipe does not exceed a
user-specified failure probability threshold. The key ideas in this paper are:
(1) to learn a surrogate predictor model from data, (2) to perform reachability
analysis using the surrogate model, and (3) to quantify the surrogate model's
incurred error using conformal inference in order to give probabilistic
reachability guarantees. We focus on learning-enabled control systems with
complex closed-loop dynamics that are difficult to model symbolically, but
where state transition pairs can be queried, e.g., using a simulator. We
demonstrate the applicability of our method on examples from the domain of
learning-enabled cyber-physical systems
- …